Probabilistic Combination of Noisy Points and Planes for RGB-D Odometry

This work proposes a visual odometry method that combines points and plane primitives, extracted from a noisy depth camera. Depth measurement uncertainty is modelled and propagated through the extraction of geometric primitives to the frame-to-frame motion estimation, where pose is optimized by weighting the residuals of 3D point and planes matches, according to their uncertainties. Results on an RGB-D dataset show that the combination of points and planes, through the proposed method, is able to perform well in poorly textured environments, where point-based odometry is bound to fail.Comment: Accepted to TAROS 201

Application of spectral and spatial indices for specific class identification in Airborne Prism EXperiment (APEX) imaging spectrometer data for improved land cover classification

Hyperspectral remote sensing's ability to capture spectral information of targets in very narrow bandwidths gives rise to many intrinsic applications. However, the major limiting disadvantage to its applicability is its dimensionality, known as the Hughes Phenomenon. Traditional classification and image processing approaches fail to process data along many contiguous bands due to inadequate training samples. Another challenge of successful classification is to deal with the real world scenario of mixed pixels i.e. presence of more than one class within a single pixel. An attempt has been made to deal with the problems of dimensionality and mixed pixels, with an objective to improve the accuracy of class identification. In this paper, we discuss the application of indices to cope with the disadvantage of the dimensionality of the Airborne Prism EXperiment (APEX) hyperspectral Open Science Dataset (OSD) and to improve the classification accuracy using the Possibilistic c–Means (PCM) algorithm. This was used for the formulation of spectral and spatial indices to describe the information in the dataset in a lesser dimensionality. This reduced dimensionality is used for classification, attempting to improve the accuracy of determination of specific classes. Spectral indices are compiled from the spectral signatures of the target and spatial indices have been defined using texture analysis over defined neighbourhoods. The classification of 20 classes of varying spatial distributions was considered in order to evaluate the applicability of spectral and spatial indices in the extraction of specific class information. The classification of the dataset was performed in two stages; spectral and a combination of spectral and spatial indices individually as input for the PCM classifier. In addition to the reduction of entropy, while considering a spectral-spatial indices approach, an overall classification accuracy of 80.50% was achieved, against 65% (spectral indices only) and 59.50% (optimally determined principal component

REVERSE DOMAIN ADAPTATION FOR INDOOR CAMERA POSE REGRESSION

Synthetic images have been used to mitigate the scarcity of annotated data for training deep learning approaches, followed by domain adaptation that reduces the gap between synthetic and real images. One such approach is using Generative Adversarial Networks (GANs) such as CycleGAN to bridge the domain gap where the synthetic images are translated into real-looking synthetic images that are used to train the deep learning models. In this article, we explore the less intuitive alternate strategy for domain adaption in the reverse direction; i.e., real-to-synthetic adaptation. We train the deep learning models with synthetic data directly, and then during inference we apply domain adaptation to convert the real images to synthetic-looking real images using CycleGAN. This strategy reduces the amount of data conversion required during the training, can potentially generate artefact-free images compared to the harder synthetic-to-real case, and can improve the performance of deep learning models. We demonstrate the success of this strategy in indoor localisation by experimenting with camera pose regression. The experimental results indicate an improvement in localisation accuracy is observed with the proposed domain adaptation as compared to the synthetic-to-real adaptation

Entropy Based Determination of Optimal Principal Components of Airborne Prism Experiment (APEX) Imaging Spectrometer Data for Improved Land Cover Classification

Hyperspectral data finds applications in the domain of remote sensing. However, with the increase in amounts of information and advantages associated, come the "curse" of dimensionality and additional computational load. The question most often remains as to which subset of the data best represents the information in the imagery. The present work is an attempt to establish entropy, a statistical measure for quantifying uncertainty, as a formidable measure for determining the optimal number of principal components (PCs) for improved identification of land cover classes. Feature extraction from the Airborne Prism EXperiment (APEX) data was achieved utilizing Principal Component Analysis (PCA). However, determination of optimal number of PCs is vital as addition of computational load to the classification algorithm with no significant improvement in accuracy can be avoided. Considering the soft classification approach applied in this work, entropy results are to be analyzed. Comparison of these entropy measures with traditional accuracy assessment of the corresponding „hardened‟ outputs showed results in the affirmative of the objective. The present work concentrates on entropy being utilized for optimal feature extraction for pre-processing before further analysis, rather than the analysis of accuracy obtained from principal component analysis and possibilistic c-means classification. Results show that 7 PCs of the APEX dataset would be the optimal choice, as they show lower entropy and higher accuracy, along with better identification compared to other combinations while utilizing the APEX dataset

Ground plane detection using an RGB-D sensor

Ground plane detection is essential for successful navigation of vision based mobile robots. We introduce a very simple but robust ground plane detection method based on depth information obtained using anRGB-Depth sensor. We present two different variations of the method: the simplest one is robust in setups where the sensor pitch angle is fixed and has no roll, whereas the second one can handle changes in pitch and roll angles. Our comparisons show that our approach performs better than the vertical disparity approach. It produces accurate ground plane-obstacle segmentation for difficult scenes, which include many obstacles, different floor surfaces, stairs, and narrow corridors.Publisher's VersionAuthor Post Prin

CONTINUOUS BIM ALIGNMENT FOR MIXED REALITY VISUALISATION

Several methods exist that can be used to perform initial alignment of Building information models (BIMs) to the real building for Mixed Reality (MR) applications, such as marker-based or markerless visual methods, but this alignment is susceptible to drift over time. The existing model-based methods that can be used to maintain this alignment have multiple limitations, such as the use of iterative processes and poor performance in environments with either too many or not enough lines. To address these issues, we propose an end-to-end trainable Convolutional Neural Network (CNN) that takes a real and synthetic BIM image pair as input to regress the 6 DoF relative camera pose difference between them directly. By correcting the relative pose error we are able to considerably improve the alignment of the BIM to the real building. Furthermore, the results of our experiments demonstrate good performance in a challenging environment and high resilience to domain shift between synthetic and real images. A high localisation accuracy of approximately 7.0 cm and 0.9° is achieved which indicates the method can be used to reduce the camera tracking drift for MR applications

A 3D Face Modelling Approach for Pose-Invariant Face Recognition in a Human-Robot Environment

Face analysis techniques have become a crucial component of human-machine interaction in the fields of assistive and humanoid robotics. However, the variations in head-pose that arise naturally in these environments are still a great challenge. In this paper, we present a real-time capable 3D face modelling framework for 2D in-the-wild images that is applicable for robotics. The fitting of the 3D Morphable Model is based exclusively on automatically detected landmarks. After fitting, the face can be corrected in pose and transformed back to a frontal 2D representation that is more suitable for face recognition. We conduct face recognition experiments with non-frontal images from the MUCT database and uncontrolled, in the wild images from the PaSC database, the most challenging face recognition database to date, showing an improved performance. Finally, we present our SCITOS G5 robot system, which incorporates our framework as a means of image pre-processing for face analysis

ENTROPY-BASED INDOOR CHANGE DETECTION USING LIDAR DATA AND A 3D MODEL

Indoor change detection is important for building monitoring, building management and model-based localization and navigation systems because the real building environment may not always be the same as the design model. This paper presents a novel indoor building change detection method based on entropy. A sequence of real LiDAR scans is acquired with a static LiDAR scanner and the pose of the LiDAR scanner for each scan is then estimated. Synthetic LiDAR scans are generated with the pose of the LiDAR scanner using the 3D model. The real LiDAR scans and synthetic LiDAR scans are sliced horizontally with a certain angular interval and the entropy of all slices of LiDAR scans is then calculated. Differenced entropy between two corresponding slices of real LiDAR scans and synthetic LiDAR scans is calculated for the classification of the changes. Each slice of real LiDAR scans will be classified into one of the four categories of changes: unchanged, moving objects, structural change and non-structural change. Experimental results show that unchanged slices and slices containing moving objects can be accurately detected, achieving 100% accuracy while non-structural and structural changes are detected with an accuracy of 98.5% and 86.3% respectively

FAST CONVERGING LIDAR-AIDED PRECISE POINT POSITIONING: A CASE STUDY WITH LOW-COST GNSS

With the growing interest in autonomous driving, accurate vehicle positioning remains an open problem, especially in urban environments. According to regulatory organisations, the vehicle positioning accuracy is required to be centimetre-level. As the most used positioning technique which provides globally referenced positioning solutions, GNSS is the fundamental component for realising real-time vehicle positioning, usually through the RTK approach. However, RTK requires a nearby reference station to enable integer ambiguity resolution for the ultra-precise carrier phase observations. In comparison, PPP makes use of State-Space Representation (SSR) corrections produced by global networks for satellite orbits and clocks to facilitate phase-based positioning. Moreover, IGS now offers Real-Time Service (RTS) to transmit such corrections. Notably, the major drawback of PPP is that it takes a long time to converge to precise solutions due to the carrier phase ambiguities being real-valued, which can be severely elongated when real-time corrections and low-cost GNSS receivers are used. In this paper, a tightly coupled positioning method is proposed, which shortens RT-PPP convergence to seconds by using lidar measurements referenced from an HD map through deep learning. The lidar measurements are generated by point cloud registration and weighted by their intensity values and geometric distributions, and are then combined with RT-PPP in an Extended Kalman-Filter (EKF), thus achieving fast convergence. Experimental results show that the proposed method achieves and maintains centimetre-level accuracy within 2 seconds using a low-cost UBLOX F9P receiver, which is a significant improvement as compared to the decimetre-level accuracy obtained from standalone RT-PPP

Estimating Depth from RGB and Sparse Sensing

We present a deep model that can accurately produce dense depth maps given an RGB image with known depth at a very sparse set of pixels. The model works simultaneously for both indoor/outdoor scenes and produces state-of-the-art dense depth maps at nearly real-time speeds on both the NYUv2 and KITTI datasets. We surpass the state-of-the-art for monocular depth estimation even with depth values for only 1 out of every ~10000 image pixels, and we outperform other sparse-to-dense depth methods at all sparsity levels. With depth values for 1/256 of the image pixels, we achieve a mean absolute error of less than 1% of actual depth on indoor scenes, comparable to the performance of consumer-grade depth sensor hardware. Our experiments demonstrate that it would indeed be possible to efficiently transform sparse depth measurements obtained using e.g. lower-power depth sensors or SLAM systems into high-quality dense depth maps.Comment: European Conference on Computer Vision (ECCV) 2018. Updated to camera-ready version with additional experiment